Physiological Predictors of Multiple Organ Dysfunction Syndrome (MODS)
Data Science and Predictive Analytics (HS650), Fall 2021
- Term Paper
- Fall 2021, DSPA (HS650)
- Name: Safa Jabri
- SID: #### - 6332
- UMich E-mail: safajb@umich.edu
- I certify that the following paper represents my own independent work and conforms with the guidelines of academic honesty described in the UMich student handbook.
Abstract
Multiple organ Dysfunction Syndrome (MODS) is a syndrome that occurs in 30-40% of trauma patients and is characterized by the dysfunction and, in acute cases,failure of more than one organ. The early detection or prediction of the occurrence of MODS can help clinicians identify patients in need of monitoring and specialized care. The goal of this project is to develop a model to predict the development of MODS in trauma patients based on early measures of immunological biomarkers and clinical assessments within the first 72h from injury.
A publicly available dataset collected through blood samples from 61 adult trauma patients at three different time points (<1h, 4–12h, 48–72h) after injury along with a label designating whether they developed MODS. The dataset was preprocessed to remove or impute missing data then randomly split into training and testing data following and 80-20 split.
Based on this dataset, we built four general linear modesl with LASSO regularization (with 10-fold cv) to predict MODS in trauma patients within 1h, within 12h, within 72h of traumatic injury based on immunological biomarkers levels collected from blood samples as well as clinical assessment scores for survival and injury severity. For each model performance was assessed based on conventional metrics (accuracy, sensitivity, specificity, confusion matrice, area under the receiver operating curve AUROC) and predictive features included in the highest performing model were identified.
This work shows preliminary evidence of the predictive power of biomarkers and clinical measures in the first 72 hours after injury in the context of MODS using simple general linear models. In order to assess the generalizability of these methods, additional data should be collected from a larger patient population.
Introduction
Multiple organ Dysfunction Syndrome (MODS) is a life-threatening syndrome whereby more than one organ exhibits signs of altered function[1] that may lead to their failure. MODS generally occurs as a complication among 30-40% of traumatic injury patients and patients admitted to intensive care units (ICU)[2], accounting for up to 50% of ICU deaths[3]. The syndrome’s complex pathological and physiological mechanisms are not fully understood, as there are multiple pathways that may lead to the failure of multiple organs[2] but it is thought to be usually caused by uncontrolled immune response leading to uncontrolled inflammation[4], similar to sepsis and systemic inflammatory response syndrome (SIRS).
Typically, MODS is assessed by clinicians by looking at trends in patients’ vital signs and examining various organs for signs of dysfunction, but early detection of the syndrome can be crucial in preventing its progression to life-threatening organ failure. A proposed way to detect MODS early, is to examine the early immune response to trauma to predict the occurrence of uncontrolled inflammation and organ dysfunction. The Golden Hour study[2,5] aimed to achieve this by analyzing the very early, within 1-hour of and up to 72 hours after trauma, response of the immune system to examine the relationship between levels of immunological biomarkers and the development of MODS (hence the name of the project).
Hypothesis
We hypothesize that immunological biomarkers detected within 72 hours after trauma can predict the development of Multiple Organ Dysfunction Syndrome (MODS).
Methods
Dataset
The analysis is based on a public dataset of immunological response measurements[6], collected by Bravo-Merodio et al. (2019)[2]. The immune and inflammatory response to trauma was collected through blood samples from 61 adult trauma patients at three different time points (<1h, 4–12h, 48–72h).
For each patient immunological and clinical parameters with the immunological variables belonging to the following categories were collected: neutrophil functional analysis (response to formyl-methionine-leucine-phenylalanine (fMLF)), monocyte response to lipopolysaccharides (LPS) stimulation, haematological, cfDNA analysis and serum analysis (cytokines and cortisol).
In addition, patients’ New Injury Severity Score (NISS), assessing the combined effects of injuries in multiply-injured patients was also recorded, as well as Population Survival (PS14) values evaluating survival and mortality after traumatic injury.
Finally a binary label indicating the development of MODS within each patient was also collected. A full list of all features included in the dataset is available in the Appendix section.
Analysis
The goal of this analysis was to build and assess the performance of machine learning (ML) models to predict labels indicating the development of MODS in patients of traumatic injury. A secondary goal is to identify features that best predict the development of MODS.
To achieve this goal the following steps were undertaken to analyze the data:
Data Pre-processing: Assessing and Handeling Missing Data
To prepare the data for analysis, we first assessed the quality of the publicly available dataset by evaluating its missingness using the R library “mi”. Patients with less than 50% of features available were first eliminated, then the rest of the missing data was imputed using the function mi().
Summary statistics of the dataset before and after imputation are provided.
Bulding a Predictive Models of MODS
In order to assess our ability to predict MODS through immunological biomarkers (within <1h, 4–12h, 48–72h) and the NISS and PS14 injury severity and survival scores, I built 4 predictive models using the linear classifier gmlnet() to assess the predictive power of the model as more information is gathered about patients throughout time. The 4 models include the following data:
Model 1: Predicting MODS with data only from the first 1h after injury (includes only immunological biomarker levels from blood samples from data collections at T1 (<1h) and no NISS or PS14 scores)
Model 2: Predicting MODS with data only from the first 12h after injury (includes only immunological biomarker levels from blood samples from data collections at T1 (<1h) and T2 (4-12h) and no NISS or PS14 scores)
Model 3: Predicting MODS with data only from the first 72h after injury (includes only immunological biomarker levels from blood samples from data collections at T1 (<1h) , T2 (4-12h) and T3 (48-72h) and no NISS or PS14 scores)
Model 4: Predicting MODS with data all of the data gathered y (includes only immunological biomarker levels from blood samples from data collections at T1 (<1h) , T2 (4-12h) and T3 (48-72h), as well as NISS and PS14 scores)
Each of the models was built following the steps outlined below:
Step 2: Model Training with LASSO Regularization
A linear algorithm with Least Absolute Shrinkage and Selection Operator (LASSO) regularization was then used to train the model on the training set.
By penalizing the sum of the absolute values of the regression coefficients, LASSO regularization allows for automatic feature selection by shrinking coefficients of inimportant features, thus reducing the dimensionality of the model. LASSO regularization parameter \(\lambda\) was selected following a 10-fold cross-validation scheme to optimize prediction performance. To reduce the variability in model performance due to the variability of lambdas selected with each run of the cross-validation, the optimal\(\lambda\) was selected by averaging lambda.1se values for 100 repeated runs of cv.glmnet()
Step 3: Model Assessment
To assess the performance of the model, we predicted the MODS labels of patients from the testing set using the trained model and reported the confusion matrix.
In addition, we plotted the Receiver-Operating Characteristic curve (ROC) and reported the Area Under the Receiver-Operating Characteristic curve (AUROC).
Overview of methods
Results
Dataset Visualization
The publicly available dataset included 61 patients between 18-90 years old. The distribution of the patients’ ages can be found in the figure below:
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 18.00 24.00 40.00 40.56 51.00 90.00
The distribution of MODS labels among the 61 patients can be seen in the figure below; 35 of the 61 patients (57.4%) developed MODS while 26 did not (42.6%)
Data Pre-processing: Assessing and Handeling Missing Data
The unprocessed dataset was missing 13.6% of its data, we can visualize missing values in the image below. We can see in the image that a few observations account for most of the missing data, as the dark boxes representing missing data stretched horizontally.
After removing patients (or observations) where 50% or more of the data is missing, we reduce the dataset from 61 to 57 observations and reduce the percentage of missing data to 10.5%
The reduced dataset (57 patients x 197 variables) was then completed via imputation. Visualization plots of some of the variables are listed below to examine their distributions, relative to the original distributions in the raw dataset:
The distribution of MODS labels among the 57 patients can be seen in the figure below; 32 of the 57 patients (56.1%) developed MODS while 25 did not (43.9%). The distribution of labels was roughly maintained.
QQ plots below show the relationship between the original and imputed distribution of the data for some examples of features. Distributions were roughly maintained after the imputation of missing data.
Predictive Models of MODS
Model 1: Predicting MODS based on data from <1h after injury
The first model is based on data collected from blood samples taken within the first hour after traumatic injury occurred. In this analysis, we examine the predictive power of this very early measurement of the immune response at predicting MODS.
The reduced dataset is composed of 57 patients (or observations) and 65 features, corresponding to 1 label as seen in the table below:
Table 1. Dataset of immunological biomarkers collected <1h after traumatic injury
Step 1: Removing Highly Corrleated Features
We calculated correlation values for all features in the model and remove highly correlated (>0.80) features as a first effort to reduce the dimentionality of the model. After removing the correlated features (shown in the correlation plot below), the dataset is reduced from 65 features to 54 features.
11 features were found to be highly correlated with other features in the model (and thus introducing little information to the model) and were therefore removed. An overview of the correlation matrix produced after highly correlated features were removed is shown below. We can see that there are no highly correlated features in the dataset anymore.
Step 2: Linear Model with LASSO Regularization
After splitting the pre-processed dataset into a training set and a test set, we used 10-fold cross validation to determine the optimal LASSO regularization parameter \(\lambda\) to minimize binomial deviance of a general linear model.
We then fit the final linear model he optimal LASSO regularization parameter \(\lambda\) (following 1se method) to the training data.
Step 3: Model Assessment
The bar plot below shows the features assigned a non-zero coefficient in the final mode. LASSO regularization shrunk the model by assigning zero coefficients to certain features. We can see that out of the 54 features initially introduced to the model, fewer than 10 variables had non-zero coefficients.
We then use the linear model to predict labels for individuals from the test set. We plot below below the confusion matrix and ROC curve, and report the AUROC and performance measures at a classification threshold of 0.5
The ROC curve shows the tradeoff between the False Positive Rate (FPR) and True Positive Rate (TPR) in the model.
The model’s prefrormance metrics can be seen blow. The model based on data for the first hour after traumatic injury shows that the model wasn’t particularly good at predicting MODS
##
## Method used: empirical
## Number of positive(s): 5
## Number of negative(s): 7
## Area under curve: 0.4286
Model 2: Predicting MODS based on data from the first 12h after injury
The first model is based on data collected from blood samples taken within the first 12 hours after traumatic injury occurred. In this analysis, we examine the predictive power of measurements of the immune response within 12h from injury at predicting MODS.
The reduced dataset is composed of 57 patients (or observations) and 129 features, corresponding to 1 label as seen in the table below:
Table 2. Dataset of immunological biomarkers collected within 12h after traumatic injury
Step 1: Removing Highly Corrleated Features
We calculated correlation values for all features in the model and remove highly correlated (>0.80) features as a first effort to reduce the dimentionality of the model. After removing the correlated features (shown in the correlation plot below), the dataset is reduced from 129 features to 107 features.
22 features were found to be highly correlated with other features in the model (and thus introducing little information to the model) and were therefore removed. An overview of the correlation matrix produced after highly correlated features were removed is shown below. We can see that there are no highly correlated features in the dataset anymore.
Step 2: Linear Model with LASSO Regularization
After splitting the pre-processed dataset into a training set and a test set, we used 10-fold cross validation to determine the optimal LASSO regularization parameter \(\lambda\) to minimize binomial deviance of a general linear model.
We then fit the final linear model he optimal LASSO regularization parameter \(\lambda\) (following 1se method) to the training data.
Step 3: Model Assessment
The bar plot below shows the features assigned a non-zero coefficient in the final mode. LASSO regularization shrunk the model by assigning zero coefficients to certain features. We can see that out of the 107 features initially introduced to the model, again fewer than 10 features had non-zero coefficients.
We then use the linear model to predict labels for individuals from the test set. We plot below below the confusion matrix and ROC curve, and report the AUROC and performance measures at a classification threshold of 0.5
The ROC curve shows the tradeoff between the False Positive Rate (FPR) and True Positive Rate (TPR) in the model.
The model’s prefrormance metrics can be seen blow. The model based on data for the first hour after traumatic injury shows that the model wasn’t particularly good at predicting MODS
##
## Method used: empirical
## Number of positive(s): 5
## Number of negative(s): 7
## Area under curve: 0.2571
Model 3: Predicting MODS based on data from the first 72h after injury
The third model is based on data collected from blood samples taken within 72 hours after traumatic injury occurred. In this analysis, we examine the predictive power of all measurements of the immune response at predicting MODS.
The reduced dataset is composed of 57 patients (or observations) and 192 features, corresponding to 1 label as seen in the table below:
Table 3. Dataset of immunological biomarkers collected within 72h after traumatic injury
Step 1: Removing Highly Corrleated Features
We calculated correlation values for all features in the model and remove highly correlated (>0.80) features as a first effort to reduce the dimentionality of the model. After removing the correlated features (shown in the correlation plot below), the dataset is reduced from 192 features to 159 features.
33 features were found to be highly correlated with other features in the model (and thus introducing little information to the model) and were therefore removed. An overview of the correlation matrix produced after highly correlated features were removed is shown below. We can see that there are no highly correlated features in the dataset anymore.
Step 2: Linear Model with LASSO Regularization
After splitting the pre-processed dataset into a training set and a test set, we used 10-fold cross validation to determine the optimal LASSO regularization parameter \(\lambda\) to minimize binomial deviance of a general linear model.
We then fit the final linear model he optimal LASSO regularization parameter \(\lambda\) (following 1se method) to the training data.
Step 3: Model Assessment
The bar plot below shows the features assigned a non-zero coefficient in the final mode. LASSO regularization shrunk the model by assigning zero coefficients to certain features. We can see that out of the 159 features initially introduced to the model, again fewer than 10 features had non-zero coefficients.
We then use the linear model to predict labels for individuals from the test set. We plot below below the confusion matrix and ROC curve, and report the AUROC and performance measures at a classification threshold of 0.5
The ROC curve shows the tradeoff between the False Positive Rate (FPR) and True Positive Rate (TPR) in the model.
The model’s prefrormance metrics can be seen blow. The model based on data for the first hour after traumatic injury shows that the model wasn’t particularly good at predicting MODS
##
## Method used: empirical
## Number of positive(s): 5
## Number of negative(s): 7
## Area under curve: 0.7429
Model 4: Predicting MODS based on all data gathered including PS14 and NISS scores
The final model is based on data collected from blood samples taken within 72 hours after traumatic injury occurred, as well as clinical assessment scores (PS14 and NISS). In this analysis, we examine the predictive power of all measurements of the immune response and assessments of the severity of injury at predicting MODS.
The reduced dataset is composed of 57 patients (or observations) and 192 features, corresponding to 1 label as seen in the table below:
Table 4. Dataset of immunological biomarkers and cliniclal assessments collected with 72h after traumatic injury
Step 1: Removing Highly Corrleated Features
We calculated correlation values for all features in the model and remove highly correlated (>0.80) features as a first effort to reduce the dimentionality of the model. After removing the correlated features (shown in the correlation plot below), the dataset is reduced from 196 features to 147.
Step 2: Linear Model with LASSO Regularization
After splitting the pre-processed dataset into a training set and a test set, we used 10-fold cross validation to determine the optimal LASSO regularization parameter \(\lambda\) to minimize binomial deviance of a general linear model.
We then fit the final linear model he optimal LASSO regularization parameter \(\lambda\) (following 1se method) to the training data.
Step 3: Model Assessment
The bar plot below shows the features assigned a non-zero coefficient in the final mode. LASSO regularization shrunk the model by assigning zero coefficients to certain features. We can see that out of the 147 features initially introduced to the model, fewer than 20 variables have non-zero coefficients.
The boxplot above showcases the distribution of patients that developed MODS and those who didn’t among 8 of the features included in the model. We can see that patients with MODS tend to have lower PS14 scores indicating that their survival likelihood tends to be evaluated lower within the first 72h after injury. Patients who develop MODS also tended to score higher on the NISS assessment, indicating a higher severity of their injuries.
We then use the linear model to predict labels for individuals from the test set. We plot below below the confusion matrix and ROC curve, and report the AUROC and performance measures at a classification threshold of 0.5
The ROC curve shows the tradeoff between the False Positive Rate (FPR) and True Positive Rate (TPR) in the model.
The model was able to achieve an accuracy of 83.3% with 1 false positive prediction and 1 false negative prediction.
##
## Method used: empirical
## Number of positive(s): 5
## Number of negative(s): 7
## Area under curve: 0.9143
Conclusions and Discussion
The early detection of MODS can facilitate the identification of traumatic injury patients in need of preventative and supportive care. In this project, we trained four linear models to predict the development of MODS among patients with traumatic injury based on early biomarkers and clinical assessments.
Models based on immunological biomarker data collected within 1h and 12 hours from injury did not show much predictive power as reflected in their perfomrance metrics. The model including immunological biomarker data collected up to 72 hours after traumatic injury however was able to better predict MODS within this patient population. These findings show evidence that the immune response within the first 12 hrs may not hold signinficant information to predict the development of MODS, but monitoring immunological biomarkers for the first 72h may preove useful in identifying indivudals at risk of developing MODS.
The linear model with LASSO regularization based on all of the data gathere from this dataset including clinical assessment score for injury severity (NISS) and survival (PS14) was able to predict the development of MODS with an accuracy of 83.3%.
Feature selection was performed using LASSO regularization to penalize the model’s complexity and shrink the model to only its most informative features. Two features played an important role in predicting the development of MODS within trauma patients, clinical measures such as PS14 and NISS scores. Other features as well relating to biomarkers outlined in the boxplots above.
The model had relatively high specificity and sensitivity (>=0.80) as it had 2 false predictions (1 false positive and 1 false negative).These findings provide additional context for clinicians interpreting the results of such a predictive model, as it provides further insights into the types of errors produced by this model.
This work shows preliminary evidence of the predictive power of biomarkers and clinical measures in the first 72 hours after injury in the context of MODS using simple general linear models.
Limitation to this work include the small sample size. In order to assess the generalizability of these methods, additional data should be collected from a larger patient population. In addition, the lack of demographic information beyond age about the patients included in the dataset and the lack of information on the types of injuries suffered limit the types of analyses that can be performed and questions that can be answered. Further studies should examine the relationships between demographic data and injury types with the development of MODS to provide more insights into this complex syndrome.
Appendix
Table 1. Raw dataset of physiological and clinical data from 61 patient (courtesy of Bravo-Merofio et al.[2,6])
References
[1] Marshall, John C. The Multiple Organ Dysfunction Syndrome. Surgical Treatment: Evidence-Based and Problem-Oriented. Zuckschwerdt, 2001. https://www.ncbi.nlm.nih.gov/books/NBK6868/.
[2] Bravo-Merodio, Laura, Animesh Acharjee, Jon Hazeldine, Conor Bentley, Mark Foster, Georgios V. Gkoutos, and Janet M. Lord. “Machine Learning for the Detection of Early Immunological Markers as Predictors of Multi-Organ Dysfunction.” Scientific Data 6, no. 1 (December 19, 2019): 328. https://doi.org/10.1038/s41597-019-0337-6.
[3] Mayr, Viktoria D., Martin W. Dünser, Veronika Greil, Stefan Jochberger, Günter Luckner, Hanno Ulmer, Barbara E. Friesenecker, Jukka Takala, and Walter R. Hasibeder. “Causes of Death and Determinants of Outcome in Critically Ill Patients.” Critical Care (London, England) 10, no. 6 (2006): R154. https://doi.org/10.1186/cc5086.
[4] Choudhary, Devendra, Hanmant Goykar, Dnyaneshwar Kalyane, Nimeet Desai, and Rakesh Kumar Tekade. “Chapter 3 - Dose, Dosage Regimen, and Dose Adjustment in Organ Failure.” In Biopharmaceutics and Pharmacokinetics Considerations, edited by Rakesh Kumar Tekade, 1:29–82. Advances in Pharmaceutical Product Development and Research. Academic Press, 2021. https://doi.org/10.1016/B978-0-12-814425-1.00004-8.
[5] Hazeldine, Jon, David N. Naumann, Emma Toman, David Davies, Jonathan R. B. Bishop, Zhangjie Su, Peter Hampson, et al. “Prehospital Immune Responses and Development of Multiple Organ Dysfunction Syndrome Following Traumatic Injury: A Prospective Cohort Study.” PLOS Medicine 14, no. 7 (July 18, 2017): e1002338. https://doi.org/10.1371/journal.pmed.1002338.
[6] In FlamUOB/NSDTraumaMODS: NSDTraumaMODS. Zenodo, https://doi.org/10.5281/zenodo.3546293 (2019).